Arbitrating an outcome of a multiplayer game session

ABSTRACT

An arbitration system is configured to receive, as a set of results data, separate instances of results data from corresponding client computing devices being used to participate in a multiplayer game session. In some examples, an instance of results data comprises input representing an indication or a selection of an identity associated with an outcome of the multiplayer game session. For instance, the input can indicate a participant identity that won the multiplayer game session. In some examples, an instance of results data comprises game data describing the overall activity in a multiplayer game session from a perspective of a player&#39;s client computing device, the game data indicating a particular participant identity as a winner of the multiplayer game session. The arbitration system is configured to use the set of results data to determine an arbitrated outcome for the multiplayer game session.

PRIORITY APPLICATION

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/418,678, filed Nov. 7, 2016, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

A multiplayer game provides players of a video game with the ability tocompete against other players either individually or in a team setting.In many multiplayer games, there are a number of scenarios where it isneither a clear nor an efficient way for a system to determine adefinitive outcome, such as a winner of the multiplayer game. Rather,existing systems and procedures apply a variety of complex algorithmsand/or rule-based evaluations to the separate performances of the gameparticipants in order to determine an outcome of a multiplayer game.Some existing systems and procedures can be costly and inefficient dueto the amount of time and computational resources that are required toapply the complex algorithms and/or the rule-based evaluations to theseparate performances to determine the outcome. In addition, someexisting systems and procedures can leave much to be desired when itcomes to the accuracy of the outcome (e.g., results).

SUMMARY

This disclosure describes an arbitration system for determining anoutcome of a multiplayer game session. For instance, the outcome cancomprise a winning participant of the multiplayer game session, a losingparticipant of the multiplayer game session, a results ranking of all orat least some of the participants of the multiplayer game session (e.g.,first place of the session, second place of the session, third place ofthe session, and so forth). The arbitration system “arbitrates” anoutcome by collecting results data from client computing devices of theparticipants and validating the results data to ensure that the resultsdata can be relied upon to determine an appropriate winner of themultiplayer game session, for example.

A participant in a multiplayer game session can comprise an individualplayer of the multiplayer game session, or alternatively, a participantcan comprise a team of players in the multiplayer game session (e.g., ateam can have two or more players). A multiplayer game session cancomprise all or a portion of any category of game that can be executedby computing devices in order to allow participants to partake in themultiplayer game session. For instance, a game can comprise an actiongame, a fighting game, a war game, a role playing game, a strategy game,a racing game, a sports game, or the like. A game can also be part oftraining software. A portion of a game that comprises the multiplayergame session can comprise a stage, a level, a round, completion of aparticular objective, completion of a particular task, a time period, amatch, and so forth.

The arbitration system is configured to receive, as a set of resultsdata, separate instances of results data from corresponding clientcomputing devices being used to participate in the multiplayer gamesession. In some examples, an instance of results data comprises inputrepresenting an indication or a selection of an identity associated withan outcome of the multiplayer game session. For instance, the input canindicate a participant identity (e.g., player identity or team identity)that won the multiplayer game session, a participant identity that lostthe multiplayer game session, and/or a performance-based ranking of allor at least some of the participant identities in the multiplayer gamesession. In some examples, an instance of results data comprises gamedata describing the overall activity in a multiplayer game session froma perspective of a player's client computing device, the game dataindicating a particular participant identity as a winner or a loser ofthe multiplayer game session. The arbitration system is configured touse the set of results data to determine an arbitrated outcome for themultiplayer game session.

In various examples described herein, an individual instance of resultsdata provides an indication of a winner of the multiplayer game session.Each participant, or in a team setting, each player, can submit aninstance of results data. Once the set of results data is received(e.g., various instances of results data are received and aggregated),the arbitration system determines whether a total number of indicationsfor a particular participant identity meets or exceeds a thresholdnumber of indications. The arbitration system sets the threshold numberof indications based on a total number of participants in themultiplayer game session and/or a mode or category of the multiplayergame (e.g., a team deathmatch mode, capture the flag mode, king of thehill mode, a role-playing category, a racing category, etc.). Inresponse to determining that the total number of indications for theparticular participant identity meets or exceeds the threshold number ofindications, the arbitration system determines that the set of resultsdata is valid and the arbitration system can store performance dataindicating that the particular identity is an arbitrated winner of themultiplayer game session. Furthermore, the arbitration system cangenerate and send, to the participating client computing devices, anotification indicating the validity of the set of results data and thewinner of the multiplayer game session. Alternatively, in response todetermining that the total number of indications for the particularparticipant identity does not meet or exceed the threshold number ofindications, the arbitration system determines that the set of resultsis invalid and cannot be relied upon to determine an arbitrated outcome.Subsequently, the arbitration system can generate and send, to theparticipating client computing devices, a notification indicating theinvalidity of the set of results data and the lack of an arbitratedoutcome.

In various examples, the arbitration system establishes a time periodduring which instances of results data that are received are deemed tobe reliable, and therefore, can be used to arbitrate an outcome. To thisend, the arbitration system can determine that an instance of resultsdata is received outside of the time period and can subsequently ignorethe instance of results data when arbitrating an outcome. For example,the instance of results data can be determined to be late if receivedafter the time period expires, and therefore, the instance of resultsdata is unreliable and not utilized in determining an arbitrated winner.In one implementation, the time period comprises a predetermined amountof time (e.g., five minutes, ten minutes, twenty minutes, thirtyminutes, etc.) after completion of the multiplayer game session.

In various examples, the arbitration system uses trustworthiness data todetermine whether the received set of results data is valid or invalid.For instance, trustworthiness data can indicate whether a particularparticipant identity has previously shown the ability and/or willingnessto provide trusted results data (e.g., correctly vote for the winner,provide accurate performance-based rankings, etc.). In oneimplementation, an indication provided by a participant can be weightedbased on a trustworthiness value established for the participant. Thearbitration system can determine the trustworthiness value for theparticipant based on a results history of the participant. Thetrustworthiness value increases as a number of arbitrated winnerscorrectly indicated (e.g., voted for) by the participant in its resultshistory increases.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key or essentialfeatures of the claimed subject matter, nor is it intended to be used asan aid in determining the scope of the claimed subject matter. The term“techniques,” for instance, may refer to system(s), method(s),computer-readable instructions, module(s), algorithms, hardware logic,and/or operation(s) as permitted by the context described above andthroughout the document.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items.

FIG. 1 is a diagram illustrating an example environment in which anarbitration system can operate to determine an outcome of a multiplayergame session based on a set of results data received from clientcomputing devices.

FIG. 2 is a diagram illustrating components of an example deviceconfigured to determine an outcome of a multiplayer game session basedon a set of results data received from client computing devices.

FIG. 3 is an example timeline illustrating a time period during whichinstances of results data can be reliably received by the arbitrationsystem.

FIG. 4 is a diagram illustrating an example of how the arbitrationsystem uses indications to determine that a set of results data receivedfrom client computing devices is valid, and therefore, can be used todetermine an arbitrated outcome of a multiplayer game session.

FIG. 5 is a diagram illustrating an example of how the arbitrationsystem uses indications and trustworthiness values to determine that aset of results data received from client computing devices is valid, andtherefore, can be used to determine an arbitrated outcome of amultiplayer game session.

FIG. 6 is a diagram illustrating an example of how the arbitrationsystem uses indications and trustworthiness values to determine that aset of results data received from client computing devices is invalid,and therefore, cannot be used to determine an arbitrated outcome.

FIG. 7 is a flow diagram of an example method for using indications todetermine whether a set of results data received from client computingdevices is valid or invalid, and arbitrating an outcome if the set ofresults are determined to be valid.

FIG. 8 is a flow diagram of an example method for establishing a timeperiod during which instances of results data can be reliably receivedby the arbitration system.

FIG. 9 is a flow diagram of an example method for adjusting a vote basedon a trustworthiness value associated with a participant identity.

DETAILED DESCRIPTION

Examples described herein provide an arbitration system that determinesan outcome of a multiplayer game session by collecting a set of resultsdata from client computing devices of the participants in themultiplayer game session and validating the set of results data toensure that the set of results data can be relied upon to correctlydetermine an outcome of the multiplayer game session (e.g., a winner, aloser, etc.). Because of the use of network-connected client computingdevices, each participant may have a different perspective on theactivity of the multiplayer game session. For example, the activity cancomprise behavior, maneuvers, earned points or a score, and/orachievements of the participants in the multiplayer game session.Consequently, to improve the accuracy with respect to arbitrating anoutcome based on input received from the client computing devices, eachclient computing device and/or participant involved in a multiplayergame session is provided with an opportunity to submit an instance ofresults data useable to arbitrate an outcome. The arbitration systemand/or another system hosting the multiplayer game session can providethe opportunity for the client computing devices to submit instances ofresults data. An instance of results data can comprise inputrepresenting an indication or a selection of a participant identityassociated with an outcome of the multiplayer game session. Additionallyor alternatively, an instance of results data can comprise game datadescribing the overall activity in a multiplayer game session from aperspective of a participant's client computing device, and thus, thegame data can indicate a particular participant identity as a winner, aloser, and/or a player rank of the multiplayer game session.

As described above, existing systems and procedures typically apply avariety of complex algorithms and/or rule-based evaluations to theseparate performances of participants in a multiplayer game session inorder to determine an outcome, which can be costly and inefficient. Thearbitration system and techniques described herein conserve computingand network resources and/or save time because the systems and servicesthat host a multiplayer game session neither have to buy nor use costlyand inefficient algorithms and/or rule-based evaluations. Rather, thesystems and services hosting a multiplayer game session can rely uponthe arbitration system described herein to perform efficient andcost-effective determination of an outcome (e.g., an arbitrated winner)of the multiplayer game session if the arbitration system can validatethe set of results data received from the client computing devices ofthe participants.

In some examples further described herein, an instance of results datadiscretely indicates an indication (e.g., a vote) for a winner or aloser of the multiplayer game session. Thus, the arbitration system isconfigured to collect and count the indications for each multiplayergame session participant that receives an indication. The arbitrationsystem is then configured to determine whether a total number ofindications for a particular participant identity that receives ahighest number of indications meets or exceeds a threshold number ofindications. Based on the determination, the arbitration systemdetermines whether the set of results data submitted by the clientcomputing devices is valid or invalid.

In various examples described herein, the arbitration system isconfigured to determine the threshold number of indications based on atotal number of participants in the multiplayer game session. Forinstance, as the total number of participants in the multiplayer gamesession increases, the likelihood of a strong consensus amongst theparticipants regarding a particular outcome is likely to decreasebecause there are more possible participants to select as the winner orthe loser. Even further, as the total number of participants in themultiplayer game session increases, the likelihood of fraudulentinstances of results data negatively affecting an arbitrated outcomewill likely decrease. Consequently, in at least some examples, thethreshold number of indications decreases as the total number ofparticipants increases. To illustrate, the arbitration system may setthe threshold number of indications at ‘thirty’ for a multiplayer gamesession with fifty participants (e.g., 60% of the participants and/orclient computing devices must select the same winner for the set ofresults data to be valid). In contrast, the arbitration system may setthe threshold number of indications at ‘fourteen’ for a multiplayer gamesession with twenty participants (e.g., 70% of the participants and/orclient computing devices must select the same winner for the set ofresults data to be valid). As yet another example, the arbitrationsystem may set the threshold number of indications at ‘eight’ for amultiplayer game session with ten participants (e.g., 80% of theparticipants and/or client computing devices must select the same winnerfor the set of results data to be valid).

Additionally or alternatively, the arbitration system can also considera mode or category of the multiplayer game when setting the thresholdnumber of indications for a multiplayer game session. Since differentmodes or categories of games generally require different player actions,it may be easier for client computing devices and/or participants,through their individual perspectives of how the multiplayer gametranspires, to evaluate performance of actions in a first mode orcategory of games compared to evaluating performance of actions in asecond mode or category of games. Thus, there is a stronger likelihoodof a consensus amongst the client computing devices and/or participantsregarding an outcome in the first mode or category of games.Consequently, the arbitration system would set a higher threshold numberof indications for a multiplayer game session in the first mode orcategory relative to a (lower) threshold number of indications set for amultiplayer game session in the second mode or category.

In some examples, the arbitration system establishes a time periodduring which instances of results data that are received are deemed tobe reliable, and therefore, can be used by the arbitration system toarbitrate an outcome. Moreover, the arbitration system can additionallyor alternatively use trustworthiness data for the participants to adjustthe weights of indications used to determine whether the received set ofresults data is valid or invalid.

Various examples, scenarios, and aspects are described below withreference to FIGS. 1-9.

FIG. 1 is a diagram illustrating an example environment 100 in which anarbitration system 102 can operate to determine an outcome of amultiplayer game session 104 based on a set of results data receivedfrom client computing devices 106(1) . . . 106(N) (where N is a positiveinteger number such as two, three, four, five, ten, fifteen, twenty,fifty, one hundred, and so forth). In at least one example, the outcomeis an arbitrated winner of the multiplayer game session 104, as furtherdescribed herein. The client computing devices 106(1) . . . 106(N)enable players to participate, individually or as a team, in themultiplayer game session 104. The multiplayer game session 104 can behosted, over a network, by a multiplayer gaming system 108 (e.g.,PLAYSTATION NOW, NINTENDO NETWORK, XBOX LIVE, etc.). That is, themultiplayer gaming system 108 can provide a multiplayer game sessionservice 110 enabling users of the client computing devices 106(1) . . .106(N) to participate in the multiplayer game session 104. As analternative, the multiplayer game session 104 can be hosted by one ofthe client computing devices 106(1) . . . 106(N) without a multiplayergaming system 108.

The arbitration system 102 comprises device(s) 112. The device(s) 112and/or other components of the arbitration system 102 can includedistributed computing resources that communicate with one another, withthe multiplayer gaming system 108, and/or with the client computingdevices 106(1) . . . 106(N) via one or more network(s) 114. In someexamples, the arbitration system 102 can be part of the multiplayergaming system 108 (e.g., resources within the system called upon toprovide arbitration services), while in other examples, the arbitrationsystem 102 is an independent system that is tasked with arbitrating anoutcome for the multiplayer game session 104. As an example, themultiplayer gaming system 108 can be a third-party system that callsupon the arbitration system 102 to reliably arbitrate an outcome.

Network(s) 114 can include, for example, public networks such as theInternet, private networks such as an institutional and/or personalintranet, or some combination of private and public networks. Network(s)114 can also include any type of wired and/or wireless network,including but not limited to local area networks (LANs), wide areanetworks (WANs), satellite networks, cable networks, Wi-Fi networks,WiMax networks, mobile communications networks (e.g., 3G, 4G, and soforth) or any combination thereof. Network(s) 114 can utilizecommunications protocols, including packet-based and/or datagram-basedprotocols such as internet protocol (IP), transmission control protocol(TCP), user datagram protocol (UDP), or other types of protocols.Moreover, network(s) 114 can also include a number of devices thatfacilitate network communications and/or form a hardware basis for thenetworks, such as switches, routers, gateways, access points, firewalls,base stations, repeaters, backbone devices, and the like.

In some examples, network(s) 114 can further include devices that enableconnection to a wireless network, such as a wireless access point (WAP).Examples support connectivity through WAPs that send and receive dataover various electromagnetic frequencies (e.g., radio frequencies),including WAPs that support Institute of Electrical and ElectronicsEngineers (IEEE) 802.11 standards (e.g., 802.11g, 802.11n, and soforth), and other standards.

In various examples, device(s) 112 can include one or more computingdevices that operate in a cluster or other grouped configuration toshare resources, balance load, increase performance, provide fail-oversupport or redundancy, or for other purposes. For instance, device(s)112 can belong to a variety of classes of devices such as traditionalserver-type devices, desktop computer-type devices, and/or mobile-typedevices. Thus, although illustrated as a single type of device—aserver-type device—device(s) 112 can include a diverse variety of devicetypes and are not limited to a particular type of device. Device(s) 112can represent, but are not limited to, server computers, desktopcomputers, web-server computers, personal computers, mobile computers,laptop computers, tablet computers, or any other sort of computingdevice.

A client computing device (e.g., one of client computing device(s)106(1) . . . 106(N)) can belong to a variety of classes of devices,which can be the same as, or different from, device(s) 112, such astraditional client-type devices, desktop computer-type devices,mobile-type devices, special purpose-type devices, embedded-typedevices, and/or wearable-type devices. Thus, a client computing devicecan include, but is not limited to, a desktop computer, a game consoleand/or a gaming device, a tablet computer, a personal data assistant(PDA), a mobile phone/tablet hybrid, a laptop computer, atelecommunication device, a computer navigation type client computingdevice such as a satellite-based navigation system including a globalpositioning system (GPS) device, a wearable device, a virtual reality(VR) device, an augmented reality (AR) device, an implanted computingdevice, an automotive computer, a network-enabled television, a thinclient, a terminal, an Internet of Things (IoT) device, a work station,a media player, a personal video recorders (PVR), a set-top box, acamera, an integrated component (e.g., a peripheral device) forinclusion in a computing device, an appliance, or any other sort ofcomputing device. In some implementations, a client computing deviceincludes input/output (I/O) interfaces that enable communications withinput/output devices such as user input devices including peripheralinput devices (e.g., a game controller, a keyboard, a mouse, a pen, avoice input device, a touch input device, a gestural input device, andthe like) and/or output devices including peripheral output devices(e.g., a display, a printer, audio speakers, a haptic output device, andthe like).

Client computing device(s) 106(1) . . . 106(N) of the various classesand device types can represent any type of computing device having oneor more processing unit(s) 116 operably connected to computer-readablemedia 118 such as via a bus 120, which in some instances can include oneor more of a system bus, a data bus, an address bus, a PCI bus, aMini-PCI bus, and any variety of local, peripheral, and/or independentbuses.

Executable instructions stored on computer-readable media 118 caninclude, for example, an operating system 122, a gaming module 124, aprofile module 126, and other modules, programs, or applications thatare loadable and executable by processing units(s) 116.

Client computing device(s) 106(1) . . . 106(N) can also include one ormore interface(s) 128 to enable communications between client computingdevice(s) 106(1) . . . 106(N) and other networked devices, such asdevice(s) 112 and/or devices of the multiplayer gaming system 108, overnetwork(s) 114. Such network interface(s) 128 can include one or morenetwork interface controllers (NICs) or other types of transceiverdevices to send and receive communications and/or data over a network.

In the example environment 100 of FIG. 1, client computing devices106(1) . . . 106(N) can use their respective gaming module 124 toconnect with one another and/or other external device(s) in order toparticipate in the multiplayer game session 104. For instance, a playercan utilize a client computing device 106(1) to access a game title.When executing the game title, the player can select a multiplayer mode,which can cause the client computing device 106(1) to connect to a gamehosting device and/or other client computing devices 106(2) . . . 106(N)over the network 114. The player can then use the client computingdevice 106(1) to register as a player, and/or as a member of a team,view and/or play in the multiplayer game session 104, and receiveupdates and/or results for the multiplayer game session 104.

The client computing device(s) 106(1) . . . 106(N) can use theirrespective profile module 132 to generate and/or receive participantprofiles, and provide the participant profiles to other client computingdevices, to game hosting devices, and/or to the device(s) 112 of thearbitration system 102. A participant profile can include one or more ofan identity of a participant (e.g., a name, a unique identifier (ID), agaming ID, etc.), a skill level of the participant, a rating for theparticipant, an age of the participant, a friends list for theparticipant, a location of the participant, etc. Participant profilescan be used to register participants for multiplayer game sessions. Insome examples, the arbitration system 102 and/or the multiplayer gamingsystem 108 maintain profile information and relevant historical data(e.g., participant identity, skill, rating, etc.) and can provide, uponrequest, the profile information to clients and/or services as requiredand as authorized (e.g., upon receiving authorized player credentials)so that the system can ensure the client or the service is trusted.

As illustrated in FIG. 1, the device(s) 112 of the arbitration system102 comprises an arbitration module 130, a data store 132, and anotification module 134. The arbitration module 130 is configured toreceive, from individual client computing devices 106(1) . . . 106(N),instances of results data 136(1) . . . 136(M) (where M is a positiveinteger number such as two, three, four, five, ten, fifteen, twenty,fifty, one hundred, and so forth). In some scenarios, not all the clientcomputing devices used to participate in the multiplayer game session104 provide an instance of results data, and thus, M (the number ofinstances submitted) may not be equal to N (the number of clientcomputing devices). In FIG. 1, for example, the arbitration module 130does not receive an instance of results data from client computingdevice 106(N), which is being used by a player to participate in themultiplayer game session 104. As described above, an instance of resultsdata can comprise input representing an indication or a selection of aparticipant identity associated with an outcome of the multiplayer gamesession 104. Additionally or alternatively, an instance of results datacan comprise game data describing the overall activity in themultiplayer game session from a perspective of a participant's clientcomputing device, and thus, the game data can indicate a participantidentity as an outcome (e.g., a winner or a loser) of the multiplayergame session 104.

The arbitration module 130 is also configured to receive sessioninformation 138 and to store the session information 138 in the datastore 132. The session information 138 can be received from the clientcomputing devices 106(1) . . . 106(N) along with the instances ofresults data 136(1) . . . 136(M) and/or as the multiplayer game session104 is being played (e.g., prior to receiving the instances of resultsdata 136(1) . . . 136(M)). Additionally or alternatively, the sessioninformation 138 can be received from the multiplayer gaming system 108,as illustrated in FIG. 1. The session information 138 can include atotal number of participants in the multiplayer game session 104, a modeor category of the multiplayer game, activity that occurs in themultiplayer game session 104 (e.g., behavior, maneuvers, earned pointsor a score, and/or achievements of the participants), and/or other datarelated to when and how the multiplayer game session 104 is conducted.

In various examples, the arbitration module 130 establishes or sets athreshold 140 for the multiplayer game session 104. The arbitrationmodule 130 can use the threshold 140 to validate the set of results datareceived from the client computing devices 106(1) . . . 106(N) (e.g.,the set of results data being based on the collective instances ofresults data 136(1) . . . 136(M) submitted for arbitration). Thethreshold 140 can be set based on the session information 138. Forinstance, the threshold 140 can comprise a threshold number ofindications required for the arbitration system 102 to determine thatthe set of results is valid results data 142 that can be relied upon todetermine an arbitrated outcome 144 (e.g., an arbitrated winningparticipant identity of the multiplayer game session 104).

The valid results data 142 can be stored in the data store 132 in avariety of forms. For example, the valid results data 142 can compriseperformance data stored in association with a winning participantidentity. The performance data can be used to determine a rating and/ora skill level for the winning participant identity. Furthermore, thenotification module 134 is configured to notify the client computingdevices 106(1) . . . 106(N) and/or the multiplayer gaming system 108 ofthe arbitrated outcome 144.

The threshold number of indications can be based on the total number ofparticipants in the multiplayer game session 104 and/or a mode orcategory of games to which the multiplayer game session 104 belongs. Thearbitration module 130 uses such session information 138 because, as thetotal number of participants in the multiplayer game session 104increases, the likelihood of a strong consensus amongst the participantsregarding a particular outcome is likely to decrease because there aremore possible participants to indicate or select (e.g., vote for) as thewinner or the loser, for example. Also, as the total number ofparticipants in the multiplayer game session 104 increases, thelikelihood of fraudulent instances of results data negatively affectingan arbitrated outcome will likely decrease. An example of a fraudulentinstance of results data may occur when a participant intentionallyvotes for another participant who clearly or obviously did not win themultiplayer game session 104, for example, because the other participantis a known acquaintance of the voting participant or because the clearor obvious winner is not liked by the voting participant. As analternative, the threshold number of indications can be based on anumber of instances of results data received (e.g., the number ‘M’ inFIG. 1).

In some examples, the arbitration module 130 receives trustworthiness(TW) data 146, which can be used to determine whether the received setof results data is valid or invalid. The trustworthiness data 146 can bestored in the data store 132, and can comprise trustworthiness value(s)148, where an individual trustworthiness value 148 correspond to aparticular participant identity. The trustworthiness value 148 indicateswhether that particular participant identity has previously shown theability and/or willingness to provide trusted results data. Forinstance, the arbitration system 102 can maintain a results history forthe particular participant identity, and the results history may revealwhether the particular participant identity has historically indicatedor selected a correct arbitrated outcome, has provided accurate andreliable performance-based rankings that positively help determine acorrected arbitrated outcome, and so forth. In one implementation, anindication or a selection provided via an instance of results data canbe weighted based on a trustworthiness value established for aparticipant. As an example, if an initially received indication orselection corresponds to a standard or baseline value of “1”, then thetrustworthiness value changes the initially received indication orselection into a weighted indication or selection that has a valuebetween a minimum weighted value such as “0” if the participant has beendetermined to be extremely untrustworthy and a maximum weighted valuesuch as “2” if the participant has been determined to be extremelytrustworthy. Consequently, a trustworthiness value 148 for a participantincreases as a number of arbitrated outcomes correctly indicated by theparticipant increases (e.g., the trustworthiness value 148 is positivelyaffected). However, if the participant incorrectly indicates aparticular outcome, then the trustworthiness value 148 decreases (e.g.,the trustworthiness value 148 is negatively affected). Examples of howthe trustworthiness values are used to validate the set of results dataare further described herein with respect to FIG. 5 and FIG. 6.

In various examples, the arbitration system 102 can be part of themultiplayer gaming system 108. In other examples, the multiplayer gamingsystem 108 is a third-party system that is external to the arbitrationsystem 102 and that hosts a multiplayer game session service 110 for themultiplayer game session 104. For instance, the third-party system canbe operated by game publisher(s), game developer(s), and/or tournamentorganizer(s). Upon completion of the multiplayer game session 104, thearbitration module 130 of the arbitration system 102 may be configuredto request the session information 138 from the third-party system. Thearbitration module 130 of the arbitration system 102 may also determinethat the third-party system is neither a trusted nor a certifiedthird-party system, and therefore, the arbitration module 130 canprovide an opportunity for the client computing devices 106(1) . . .106(N) to report and/or send the instances of results data 136(1) . . .136(M) indicating an outcome of the multiplayer game session 104. Forexample, the arbitration module 130 can generate a prompt via a userinterface that enables the players to submit a selection or anindication for an outcome such as a winner of the multiplayer gamesession 104. In another example, a dedicated game server can act as asource of truth, as it is tracking the true actions of each player inthe multiplayer game (e.g., when player A kills player B, the dedicatedgame server itself knows and tracks the fact that player A kills playerB). In yet another example, if a dedicated game server is not available,and the multiplayer game or match is peer-to-peer with one of the clientdevices designated as a host, then the potentially insecure set of gameclients can act as sources of truth. For instance, when player A killsplayer B, the individual game clients keep track of the fact that playerA kills player B, and at the end of the game or match, the aggregatedresults are counted and each client reports results. In this case, mostof the players will not tamper with their game clients to alter theresults prior to reporting, and therefore, malicious actors can bedetected and identified quickly so that their trustworthiness score canbe lowered.

FIG. 2 is a diagram illustrating components of an example device 200configured to determine an outcome of a multiplayer game session, suchas multiplayer game session 104, based on a set of results data receivedfrom client computing devices, such as client computing devices 106(1) .. . 106(N). Device 200 can represent one of device(s) 112. Device 200includes one or more processing unit(s) 202, computer-readable media204, communication interface(s) 206. The components of device 200 areoperatively connected, for example, via a bus, which can include one ormore of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCIbus, and any variety of local, peripheral, and/or independent buses.

As used herein, processing unit(s), such as processing unit(s) 202and/or processing unit(s) 116, can represent, for example, a CPU-typeprocessing unit, a GPU-type processing unit, a field-programmable gatearray (FPGA), another class of digital signal processor (DSP), or otherhardware logic components that may, in some instances, be driven by aCPU. For example, and without limitation, illustrative types of hardwarelogic components that can be used include Application-SpecificIntegrated Circuits (ASICs), Application-Specific Standard Products(ASSPs), System-on-a-Chip Systems (SOCs), Complex Programmable LogicDevices (CPLDs), etc.

As used herein, computer-readable media, such as computer-readable media204 and/or computer-readable media 118, can store instructionsexecutable by the processing unit(s). Computer-readable media can alsostore instructions executable by external processing units such as by anexternal CPU, an external GPU, and/or executable by an externalaccelerator, such as an FPGA type accelerator, a DSP type accelerator,or any other internal or external accelerator. In various examples, atleast one CPU, GPU, and/or accelerator is incorporated in a computingdevice, while in some examples one or more of a CPU, GPU, and/oraccelerator is external to a computing device.

Computer-readable media can include computer storage media and/orcommunication media. Computer storage media can include one or more ofvolatile memory, nonvolatile memory, and/or other persistent and/orauxiliary computer storage media, removable and non-removable computerstorage media implemented in any method or technology for storage ofinformation such as computer-readable instructions, data structures,program modules, or other data. Thus, computer storage media includestangible and/or physical forms of media included in a device and/orhardware component that is part of a device or external to a device,including but not limited to random-access memory (RAM), staticrandom-access memory (SRAM), dynamic random-access memory (DRAM), phasechange memory (PCM), read-only memory (ROM), erasable programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM), flash memory, compact disc read-only memory (CD-ROM),digital versatile disks (DVDs), optical cards or other optical storagemedia, magnetic cassettes, magnetic tape, magnetic disk storage,magnetic cards or other magnetic storage devices or media, solid-statememory devices, storage arrays, network attached storage, storage areanetworks, hosted computer storage or any other storage memory, storagedevice, and/or storage medium that can be used to store and maintaininformation for access by a computing device.

In contrast to computer storage media, communication media can embodycomputer-readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave, or othertransmission mechanism. As defined herein, computer storage media doesnot include communication media. That is, computer storage media doesnot include communications media consisting solely of a modulated datasignal, a carrier wave, or a propagated signal, per se.

Communication interface(s) 206 can represent, for example, networkinterface controllers (NICs) or other types of transceiver devices tosend and receive communications over a network.

In the illustrated example, computer-readable media 204 includes thedata store 132. In some examples, data store 132 includes data storagesuch as a database, data warehouse, or other type of structured orunstructured data storage. In some examples, data store 132 includes acorpus and/or a relational database with one or more tables, indices,stored procedures, and so forth to enable data access including one ormore of hypertext markup language (HTML) tables, resource descriptionframework (RDF) tables, web ontology language (OWL) tables, and/orextensible markup language (XML) tables, for example.

Data store 132 can store data for the operations of processes,applications, components, and/or modules stored in computer-readablemedia 204 and/or executed by processing unit(s) 202 and/oraccelerator(s). For instance, in some examples, data store 132 can storemultiplayer game session data 208 (e.g., session information 138),profile data 210, and/or third-party system data 212. The multiplayergame session data 208 can comprise a total number of participants in themultiplayer game session 104, a mode or category of the multiplayergame, activity that occurs in the multiplayer game session 104 (e.g.,behavior, maneuvers, earned points or a score, and/or achievements ofthe participants), and/or other data related to when and how themultiplayer game session 104 is conducted or hosted.

Examples of profile data 210 include, but are not limited to, aparticipant identity (ID) 214, voting history data 216, atrustworthiness value 218, performance data 220, and/or skills data 222.For example, performance data 220 can include data associated witharbitrated outcomes (e.g., wins) for a corresponding participant ID 214.Moreover, the performance data 220 can be used to determine skills data222 for the participant ID 214 (e.g., a skill rating or ranking in aparticular game mode or category). The profile data 210 can be stored inthe form of a player or a team profile.

Third-party system data 212 can include data indicating whether anoperator of the third-party system and the gaming services it providesis trusted and/or certified. As described above, if the third-partysystem is neither a trusted nor a certified system, then the arbitrationsystem 102 may be required to arbitrate the outcome via the instances ofresults data 136(1) . . . 136(M) received from the client computingdevices 106(1) . . . 106(N), rather than rely on results data determinedby the third-party system that is neither trusted nor certified.

Alternately, some or all of the above-referenced data can be stored onseparate memories 224 on board one or more processing unit(s) 202 suchas a memory on board a CPU-type processor, a GPU-type processor, anFPGA-type accelerator, a DSP-type accelerator, and/or anotheraccelerator. In the illustrated example of FIG. 2, computer-readablemedia 204 also includes operating system 226 and application programminginterface(s) 228 configured to expose the functionality and the data ofthe device(s) 112 (e.g., example device 200) to external devicesassociated with, for example, game publishers, game developers,tournament organizers, and/or the participants (e.g., client computingdevices 106(1) . . . 106(N)). Additionally, computer-readable media 204includes one or more modules such as an arbitration module 130 and anotification module 134, although the number of illustrated modules isjust an example, and the number can vary higher or lower. That is,functionality described herein in association with the illustratedmodules can be performed by a fewer number of modules or a larger numberof modules on one device or spread across multiple devices.

As described above, in some examples, the arbitration system 102establishes a time period during which instances of results data thatare received are deemed to be reliable, and therefore, can be used toarbitrate an outcome of a multiplayer game session 104. FIG. 3 is anexample timeline 300 illustrating a time period during which instancesof results data can be reliably received by the arbitration system 102.The horizontal axis in FIG. 3 represents time, as illustrated. Asreferenced by 302, the start of the multiplayer game session occurs at afirst time (e.g., time t₁). As referenced by 304, the completion of themultiplayer game session occurs at a second time (e.g., time t₂). In atleast one implementation, the period of time during which clientcomputing devices 106(1) . . . 106(N) can report reliable results (e.g.,reliable indications or selections) begins at the time (e.g., time t₂)when the multiplayer game session is completed and the period of timeends, as referenced by 306, after a predetermined amount of time (e.g.,a pre-set threshold amount of time) expires at a third time (e.g., timet₃). Accordingly, the timeline in FIG. 3 illustrates a reliable resultsreporting period 308 that begins at the time when the multiplayer gamesession is completed and ends after a predetermined amount of timeexpires (e.g., the duration of the reliable results reporting period 308is equal to the predetermined amount of time).

Consequently, indications received by the arbitration system 102 duringthe reliable results reporting period 308 are deemed to be reliableindications 310 useable by the arbitration system 102 to determinewhether a set of results data is valid. In contrast, indicationsreceived by the arbitration system 102 outside the reliable resultsreporting period 308 (e.g., after the predetermined time period expires)are deemed to be unreliable indications 312 that are not used by thearbitration system 102 to determine whether a set of results data isvalid (e.g., late instances of results data are ignored when arbitratingan outcome). The predetermined time period comprises a threshold amountof time such as five minutes, ten minutes, twenty minutes, thirtyminutes, and so forth.

FIG. 4 is a diagram 400 illustrating an example of how the arbitrationmodule 130 of the arbitration system 102 uses indications to determinethat a set of results data received from client computing devices 106(1). . . 106(N) is valid, and therefore, can be used to determine anarbitrated outcome of a multiplayer game session 402. In this example,and for ease of discussion, the multiplayer game session 402 comprisesten participants, and the participant identities of the ten participantsstored in the data store 132 and/or the profile module(s) 126 are “ID1”, “ID 2”, “ID 3”, “ID 4”, “ID 5”, “ID 6”, “ID 7”, “ID 8”, “ID 9”, and“ID 10”.

In accordance with FIG. 1 and FIG. 2, the arbitration module 130 isconfigured to receive instances of results data 136(1) . . . 136(M) fromat least some of the client computing devices 106(1) . . . 106(N). Anindividual instance of results data may indicate an indicationassociated with an outcome, such as a winner of the multiplayer gamesession 402. The arbitration module 130 is further configured to set athreshold number of indications. The threshold number of indications canbe based on the total number of participants in the multiplayer gamesession 402 (e.g., ten) and/or on a mode or category of the multiplayergame. In this example, the arbitration module 130 sets the threshold 404at six indications such that six of the ten participant IDs mustindicate or select (e.g., vote for) the same winning participant ID forthe set of results data to be valid.

As shown, the arbitration module 130 can sort or batch the indicationsreceived based on participant identities, and in this example, threedifferent participant IDs received indications. That is, this exampleincludes a first batch of indications 406 for participant ID 2, a secondbatch of indications 408 for participant ID 3, a third batch ofindications 410 for participant ID 8. Each batch of indications includesa list including the participant ID 412 that submits the results withthe indication and a tally 414 (e.g., a vote count). Consequently, batch406 of FIG. 4 illustrates that “ID 1”, “ID 2”, “ID 4”, “ID 7”, “ID 8”,and “ID 10” indicated that participant “ID 2” is the winner, and thetotal tallied number of indications is six. Further, batch 408 of FIG. 4illustrates that “ID 3” indicated that he or she (or his/her own team)is the winner, and the total tallied number of indications is one.Finally, batch 410 of FIG. 4 illustrates that “ID 5” and “ID 9”indicated that participant ID 8 is the winner, and the total talliednumber of indications is two. Note that “ID 6” did not provide anindication, or the indication from “ID 6” was late, and therefore notrelied upon by the arbitration module 130 when validating the set ofresults data.

Based on the tallied batches of indications, the arbitration module 130,in this example, determines that participant ID 2 received the mostindications and that the set of results data received is valid becausethe total number of indications participant ID 2 receives (e.g., sixindications) is greater than or equal to the threshold number ofindications (e.g., six indications). Consequently, the arbitrationmodule 130 determines that participant ID 2 is the arbitrated winner ofthe multiplayer game session 402, and performance data 220 associatedwith this win can subsequently be stored in the data store 132.Moreover, the notification module 134 can generate and send anotification indicating that participant ID 2 is the arbitrated winnerof the multiplayer game session 402.

In some examples, the threshold number of indications is increased ordecreased by the arbitration module 130 based on a reliability ratio.The reliability ratio includes a total number of reliable indications(e.g., reliable indications 310 in FIG. 3) received from the clientcomputing devices over a total number of identities participating in themultiplayer game session. Consequently, the threshold number ofindications can increase based on the reliability ratio increasing, andthe threshold number of indications can decrease based on thereliability ratio decreasing.

FIG. 5 is a diagram 500 illustrating an example of how the arbitrationmodule 130 of the arbitration system 102 uses indications andtrustworthiness values 148 to determine that a set of results datareceived from client computing devices 106(1) . . . 106(N) is valid, andtherefore, can be used to determine an arbitrated outcome of amultiplayer game session 502. In this example, and for ease ofdiscussion, the multiplayer game session 502 again comprises tenparticipants, and the participant identities of the ten participantsstored in the data store 132 and/or the profile module(s) 126 are “ID1”, “ID 2”, “ID 3”, “ID 4”, “ID 5”, “ID 6”, “ID 7”, “ID 8”, “ID 9”, and“ID 10”.

In accordance with FIG. 1 and FIG. 2, the arbitration module 130 isconfigured to receive instances of results data 136(1) . . . 136(M) fromat least some of the client computing devices 106(1) . . . 106(N). Anindividual instance of results data may provide an indication or aselection associated with an outcome, such as a winner of themultiplayer game session 502. The arbitration module 130 is furtherconfigured to set a threshold number of indications based on the totalnumber of participants in the multiplayer game session 502 (e.g., ten)and/or on a mode or category of the multiplayer game. In this example,the arbitration module 130 again sets the threshold 504 at sixindications such that a winning participant ID needs six indications forthe set of results data to be valid.

Similar to the discussion above with respect to FIG. 4, the arbitrationmodule 130 can sort or batch the indications initially received based onparticipant identities. In this example, only one of the participant IDsthat receives indications is illustrated, as batch of indications 506(e.g., for participant ID 2). The batch of indications 506 similarlyincludes a list including the participant ID 508 that provides theresults with the indication. Consequently, the batch of indications 506of FIG. 5 illustrates that “ID 1”, “ID 2”, “ID 4”, “ID 7”, and “ID 8”indicated that participant “ID 2” is the winner, and the total talliednumber of initially received indications is five, which does not meet orexceed the threshold 504 number of indications of six required tovalidate the set of results (e.g., a sixth indications is not initiallyreceived in the set of results data). However, in the example of FIG. 5,the initially received indications are weighted, or adjusted, based ontrustworthiness values 510 (e.g., trustworthiness values 148) associatedwith the participant IDs, to produce weighted indications.

In at least one example (and the example illustrated in FIG. 5), if aninitially received indication is associated with a standard value of“1”, then the trustworthiness value may itself comprise the weightedindication. Accordingly, FIG. 5 illustrates that the indication receivedfrom participant “ID 1” is adjusted by a trustworthiness value toprovide a weighted indication of “1” (1*1=1). The indication receivedfrom participant “ID 2” is adjusted by a trustworthiness value toprovide a weighted indication of “1.2” (1*1.2=1.2). The indicationreceived from participant “ID 4” is adjusted by a trustworthiness valueto provide a weighted indication of “0.7” (1*0.7=0.7). The indicationreceived from participant “ID 7” is adjusted by a trustworthiness valueto provide a weighted indication of “1.5” (1*1.5=1.5). And theindication received from participant “ID 8” is adjusted by atrustworthiness value to provide a weighted indication of “2.0”(1*2.0=2.0). Looking at the trustworthiness values, it can be deducedthat participant “ID 8” (e.g., with a trustworthiness value of 2.0) hashistorically provided more trustworthy results data compared to thatprovided by participant “ID 4” (e.g., with a trustworthiness value of0.7).

Summing up the weighted indications provides a weighted indication tallyof “6.4”, which is greater than or equal to the threshold number ofindications (e.g., six). Consequently, based on the tallied weightedindications, the arbitration module 130 determines that the set ofresults data is valid in light of the trustworthiness values because thetotal number of weighted indications for participant ID 2 (e.g., 6.4votes) is greater than or equal to the threshold number of indications(e.g., six votes), even though the initially received number ofindications (prior to the adjustments and weights) does not meet orexceed the threshold. Consequently, the arbitration module 130 uses thetrustworthiness values 510 to validate the set of results and todetermine that participant ID 2 is the arbitrated winner of themultiplayer game session 502, and performance data 220 associated withthis win can subsequently be stored in the data store 132. Moreover, thenotification module 134 can generate and send a notification indicatingthat participant ID 2 is the arbitrated winner of the multiplayer gamesession 502.

As shown above, the trustworthiness values 510 affect the initiallyreceived indications or selections used to determine whether thereceived results data is valid and/or to determine an arbitratedoutcome. The trustworthiness values 510 are used so that results inputfrom trustworthy participants have a greater effect on the arbitratedoutcome than results input from untrustworthy participants. Thetrustworthiness values 510 can be determined based on a participant'sresults-providing history. For example, a participant may be assigned aninitially neutral trustworthiness value (e.g., “1”) if the participantis a new player to the arbitration system 102 and has no history. Thetrustworthy value can then be incremented or decremented, over time andas instances of results data are received, based on whether theparticipant shows the ability and/or willingness to indicate or selectthe correct outcome (e.g., correctly vote for the winner). Consequently,a trustworthiness value increases as a number of arbitrated outcomescorrectly indicated increases. In some examples, a trustworthy value canincrease for a trustworthy participant until it hits a defined maximumvalue (e.g., “2” such that one initial indication is actually counted astwo, “3” such that one initial indication is actually counted as three,etc.). Similarly, a trustworthy value can decrease for an untrustworthyparticipant until it hits a defined minimum value (e.g., “0” such thatone initial indication is not counted at all, “0.5” such that oneinitial indication is actually only counted as a half of an indication,etc.).

In additional or alternative examples, the arbitration module 130 cantrack the results history and determine a ratio (e.g., percentage) ofcorrect indications to a total number of indications submitted by aparticipant. This approach can be implemented once the participant meetsa minimum number of total indications such as ten, twenty, fifty, etc.If the ratio indicates the participant is correct 100% of the time, theparticipant is associated with a predetermined maximum trustworthinessvalue (e.g., 2.0, 3.0, etc.). If the ratio indicates the participant iscorrect 0% of the time, the participant is associated with apredetermined minimum trustworthiness value (e.g., 0, 0.5, etc.). Anypercentage in between 0% and 100% can be normalized or a mapped to valuebetween the predetermined minimum and maximum trustworthiness values.

FIG. 6 is a diagram 600 illustrating an example of how the arbitrationmodule 130 of the arbitration system uses indications andtrustworthiness values 148 to determine that a set of results datareceived from client computing devices is 106(1) . . . 106(N) invalid,and therefore, cannot be used to determine an arbitrated outcome of amultiplayer game session 602. In this example, and for ease ofdiscussion, the multiplayer game session 602 again comprises tenparticipants, and the participant identities of the ten participantsstored in the data store 132 and/or the profile module(s) 126 are “ID1”, “ID 2”, “ID 3”, “ID 4”, “ID 5”, “ID 6”, “ID 7”, “ID 8”, “ID 9”, and“ID 10”.

In accordance with FIG. 1 and FIG. 2, the arbitration module 130 isconfigured to receive instances of results data 136(1) . . . 136(M) fromat least some of the client computing devices 106(1) . . . 106(N). Anindividual instance of results data may provide an indication or aselection associated with an outcome, such as a winner of themultiplayer game session 602. The arbitration module 130 is furtherconfigured to set a threshold number of indications based on the totalnumber of participants in the multiplayer game session 602 (e.g., ten)and/or on a mode or category of the multiplayer game. In this example,the arbitration module 130 again sets the threshold 604 at sixindications such that a winning participant ID needs six indications forthe set of results data to be valid.

Similar to the discussion above with respect to FIG. 4 and FIG. 5, thearbitration module 130 can sort or batch the indications initiallyreceived based on participant identities. In this example, only one ofthe participant IDs that receives indications is illustrated, as batchof indications 606 (e.g., for participant ID 2). The batch ofindications 606 similarly includes a list including the participant ID608 that provides the results with an indication. Consequently, thebatch of indications 606 of FIG. 6 illustrates that “ID 1”, “ID 2”, “ID4”, “ID 7”, “ID 8”, “ID 9”, and “ID 10” indicated that participant “ID2” is the winner, and the total tallied number of initially receivedindications is seven, which is greater than the threshold 604 number ofindications of six required to validate the set of results. Similar toFIG. 5, in the example of FIG. 6, the initially received indications areweighted, or adjusted, based on trustworthiness values 610 (e.g.,trustworthiness values 148) associated with the participant IDs, toproduce weighted indications.

FIG. 6 illustrates that the indication received from participant “ID 1”is adjusted by a trustworthiness value to provide a weighted indicationof “0” (1*0=0). The indication received from participant “ID 2” isadjusted by a trustworthiness value to provide a weighted indication of“1.0” (1*1.0=1.0). The indication received from participant “ID 4” isadjusted by a trustworthiness value to provide a weighted indication of“0.7” (1*0.7=0.7). The indication received from participant “ID 7” isadjusted by a trustworthiness value to provide a weighted indication of“1.3” (1*1.3=1.3). The indication received from participant “ID 8” isadjusted by a trustworthiness value to provide a weighted indication of“0.8” (1*0.8=0.8). The indication received from participant “ID 9” isadjusted by a trustworthiness value to provide a weighted indication of“0.3” (1*0.3=0.3). And the indication received from participant “ID 10”is adjusted by a trustworthiness value to provide a weighted indicationof “0.5” (1*0.5=0.5).

Summing up the weighted indications in the example of FIG. 6 provides aweighted tally of “4.6”, which is not greater than or equal to thethreshold number of indications (e.g., six). Consequently, based on thetallied weighted indications, the arbitration module 130 determines thatthe set of results data is invalid in light of the trustworthinessvalues because the total number of weighted indications for participantID 2 (e.g., 4.6) is not greater than or equal to the threshold number ofindications (e.g., six), even though the initially received number ofindications (prior to the adjustments and weights) does meet or exceedthe threshold. Consequently, the arbitration module 130 uses thetrustworthiness values 612 to determine that the set of results areinvalid.

FIGS. 7-9 represent example processes in accordance with variousexamples from the description of FIGS. 1-6. The example operations shownin FIGS. 7-9 can be implemented on or otherwise embodied in one or moredistributed computing resources such as the device(s) 112 of thearbitration system 102, devices of the multiplayer gaming system 108,and/or the client computing devices 106(1) . . . 106(N). In variousexamples, example functions can be implemented via user interface(s)served by the arbitration system 102 and/or the host gaming system 108and accessed by the client computing devices 106(1) . . . 106(N), or assoftware running on the arbitration system 102 and/or the host gamingsystem 108. For the sake of illustration, the example processes aredescribed below with reference to processing unit 202 in device 200.However, other processing unit(s) and/or other components associatedwith other device(s) described herein can carry out operation(s) ofdescribed example processes.

The order in which the operations are described in each example flowdiagram is not intended to be construed as a limitation, and any numberof the described operations can be combined in any order and/or inparallel to implement each process. Moreover, the operations in each ofFIGS. 7-9 can be implemented in hardware, software, and/or a combinationthereof. In the context of software, the operations representcomputer-executable instructions that, when executed by one or moreprocessing units, cause one or more processing units to perform therecited operations. For example, modules and other components describedherein can be stored in a computer-readable media and executed by atleast one processing unit to perform the described operations. In thecontext of hardware, the operations can represent logic functionsimplemented in circuitry (e.g., datapath-control andfinite-state-machine sequencing functions).

FIG. 7 is a flow diagram of an example method 700 for using votes todetermine whether a set of results data received from client computingdevices is valid or invalid, and arbitrating an outcome if the set ofresults are determined to be valid.

At 702, processing unit 202 receives session information. For instance,the session information can be received from client computing devicesparticipating in a multiplayer game session and/or from a multiplayergaming system hosting the multiplayer game session or providing aservice to the multiplayer game session. The session information caninclude a total number of participants in the multiplayer game session,a mode or category of the multiplayer game, activity that occurs in themultiplayer game session (e.g., behavior, maneuvers, earned points or ascore, and/or achievements of the participants), and/or other datarelated to when and how the multiplayer game session is conducted.

At 704, processing unit 202 sets a threshold used to validate a set ofresults data. The threshold can comprise a threshold number ofindications required to validate the set of results data and todetermine an arbitrated outcome. In some examples, the threshold numberof indications can be based on the total number of participants in themultiplayer game session. In some examples, the threshold number ofindications can be based on the mode or category of multiplayer game. Insome examples, the threshold number of indications can be based on thereliability ratio described above.

At 706, processing unit 202 receives the set of results data. The set ofresults data comprises individual instances of results data receivedfrom individual ones of the client computing devices participating inthe multiplayer game session. In some examples, an instance of resultsdata discretely indicates or selects (e.g., votes for) a particularparticipant identity as the winning identity.

At 708, processing unit 202 determines whether a number of indications aparticipant identity receives via the set of results data is greaterthan or equal to the threshold number of indications. If “no” at 708,the process proceeds to 710 where the processing unit 202 determines theset of results is invalid. Then, at 712, the processing unit 202 cangenerate and send a notification, to the client computing devices forexample, indicating the set of results is invalid and the inability ofthe arbitration system 102 to determine an arbitrated outcome (e.g., anarbitrated winner).

If “yes” at 708, the process proceeds to 714 where the processing unit202 determines the set of results is valid. Then, at 716, the processingunit 202 stores data associated with an arbitrated outcome supported bythe valid set of results. And, at 718, the processing unit 202 cangenerate and send a notification, to the client computing devices forexample, indicating the set of results is valid and the arbitratedoutcome (e.g., an identity of the arbitrated winner) determined by thearbitration system 102. In some examples, individual client computingdevices and/or other services have the discretion to decide whether thearbitrated outcome is accepted. For instance, the processing unit 202can report, along with the arbitrated outcome, an overalltrustworthiness score (e.g., on a scale from 0 to 1 where 0 iscompletely untrustworthy and 1 is fully trustworthy). That way, aservice such as a matchmaking/skill service can determine on its ownwhether to consume the arbitrated outcome data for standard skillcalculations. Or a service such as a tournament service can determine onits own whether to consume the arbitrated outcome to award tournamentprizes.

FIG. 8 is a flow diagram of an example method 800 for establishing atime period during which instances of results data can be reliablyreceived by the arbitration system.

At 802, processing unit 202 determines that a reporting period for amultiplayer game session has started. For instance, the reporting periodcan begin at, or approximately at (e.g., within thirty seconds, within aminute, etc.), the time the multiplayer game session has ended.

At 804, processing unit 202 receives reliable instances of results datafor the multiplayer game session.

At 806, processing unit 202 determines that the reporting period for themultiplayer game session has ended. As described above, the reportingperiod can be a predetermined amount of time (e.g., five minutes, tenminutes, fifteen minutes, thirty minutes, etc.).

At 808, processing unit 202 receives unreliable instance(s) of resultsdata for the multiplayer game session (after the reporting period hasexpired).

At 808, processing unit 202 uses the reliable instances of results datafor the multiplayer game session to determine whether the set of resultsdata is valid and can be used to arbitrate an outcome. Moreover, theprocessing unit ignores, or does not use, the unreliable instance(s) ofresults data (e.g., received late or outside the reporting period) todetermine whether the set of results data is valid and can be used toarbitrate an outcome. Operation 808 can be implemented in associationwith operation 708 from FIG. 7.

FIG. 9 is a flow diagram of an example method 900 for adjusting anindication based on a trustworthiness value associated with aparticipant identity.

At block 902, processing unit 202 tracks results data submitted by aparticipant to generate a results history for the participant.

At block 904, processing unit 202 determines a trustworthiness value forthe participant based on the results history.

At block 906, processing unit 202 stores the trustworthiness value inassociation with an identity of the participant.

At block 908, processing unit 202 uses the trustworthiness value toadjust a standard or baseline indication value (e.g., standard valueequal to one) initially received and to generate a weighted indication.

At block 910, processing unit 202 uses the weighted indication forresults validation purposes. Operation 910 can be implemented inassociation with operation 708 from FIG. 7.

At block 912, processing unit 202 determines whether the indication ofthe participant correctly selected the arbitrated outcome (ifdetermined). Then the process returns to operation 902 where the resultshistory can be updated based on the determination of operation 912.Consequently, the participant's trustworthiness value increases as thenumber of correct indications of an arbitrated outcome increases.

EXAMPLE CLAUSES

Example Clause A, a system comprising: one or more processing units; anda computer-readable medium having encoded thereon computer-executableinstructions to configure the one or more processing units to: receive,as a set of results data, a plurality of instances of results data froma corresponding plurality of computing devices, wherein an individualinstance of results data provides an indication of an outcome for anindividual identity of a plurality of identities participating in amultiplayer game session; determine that a total number of indicationsfor an identity is greater than or equal to a threshold number ofindications, wherein the threshold number of indications is based atleast in part on a total number of the plurality of identitiesparticipating in the multiplayer game session; and in response todetermining that the total number of indications is greater than orequal to the threshold number of indications, determine that the set ofresults data is valid, and store, in association with the identity andbased at least in part on the valid set of results data, data indicatingthat the identity is an arbitrated winner of the multiplayer gamesession.

Example Clause B, the system of Example Clause A, wherein thecomputer-executable instructions further configure the one or moreprocessing units to, in response to determining that the total number ofindications is not greater than or equal to the threshold number ofindications, generate a notification indicating that the set of resultsdata is invalid data that is unable to be used to determine thearbitrated winner of the multiplayer game session.

Example Clause C, the system of Example Clause A or Example Clause B,wherein the computer-executable instructions further configure the oneor more processing units to: determine that at least one computingdevice provides late results data received outside of a predeterminedtime period; and ignore the late results data when determining that theset of results data received from the plurality of computing devicesindicates the total number of indications for the identity is greaterthan or equal to the threshold number of indications.

Example Clause D, the system of Example Clause C, wherein thepredetermined time period comprises a threshold amount of time aftercompletion of the multiplayer game session.

Example Clause E, the system of any one of Example Clauses A through D,wherein the threshold number of indications increases as a reliabilityratio increases, wherein the reliability ratio comprises a total numberof reliable indications received from the plurality of computing devicesover a total number of identities participating in the multiplayer gamesession.

Example Clause F, the system of any one of Example Clauses A through E,wherein the computer-executable instructions further configure the oneor more processing units to expose the performance data to a pluralityof third-party systems via one or more application programminginterfaces (APIs).

Example Clause G, the system of any one of Example Clauses A through F,wherein the computer-executable instructions further configure the oneor more processing units to determine the threshold number ofindications based at least in part on a mode or category of themultiplayer game session.

Example Clause H, the system of any one of Example Clauses A through G,wherein the computer-executable instructions further configure the oneor more processing units to use the data to determine a skill rating ofthe identity.

Example Clause I, the system of any one of Example Clauses A through H,wherein the computer-executable instructions further configure the oneor more processing units to: determine that a third-party system thathosts a service for the multiplayer game session is not a trustedsystem; and provide an opportunity for the plurality of computingdevices to send the plurality of instances of results data based, atleast in part, on the determination that the third-party system is notthe trusted system.

Example Clause J, the system of any one of Example Clauses A through I,wherein the computer-executable instructions further configure the oneor more processing units to: access trustworthiness data indicatingtrustworthiness values for identities that provide the plurality ofinstances of results data; and weight the indications based at least inpart on the trustworthiness values.

Example Clause K, the system of Example Clause J, wherein thecomputer-executable instructions further configure the one or moreprocessing units to determine a trustworthiness value based at least inpart on a results history of a particular identity.

Example Clause L, the system of Example Clause K, wherein thetrustworthiness value increases as a number of arbitrated winnerscorrectly indicated by the particular identity increases.

While Example Clauses A through L are described above with respect to asystem, it is understood in the context of this document that thesubject matter of Example Clauses A through L can also be implemented bya device, via a computer-implemented method, and/or viacomputer-readable storage media.

Example Clause M, a method, comprising: receiving, as a set of resultsdata, a plurality of instances of results data from a correspondingplurality of computing devices, wherein an individual instance ofresults data discretely indicates an indication of an outcome for anindividual identity of a plurality of identities participating in amultiplayer game session; determining, by one or more processing units,that a total number of indications for an identity is greater than orequal to a threshold number of indications, wherein the threshold numberof indications is based at least in part on a total number of theplurality of identities participating in the multiplayer game session;and in response to determining that the total number of indications isgreater than or equal to the threshold number of indications,determining that the set of results data is valid, and storing, inassociation with the identity and based at least in part on the validset of results data, data indicating that the identity is an arbitratedwinner of the multiplayer game session.

Example Clause N, the method of Example Clause M, further comprising:determining that at least one computing device provides late resultsdata received outside of a predetermined time period; and ignoring thelate results data when determining that the set of results data receivedfrom the plurality of computing devices indicates the total number ofindications for the identity is greater than or equal to the thresholdnumber of indications.

Example Clause O, the method of Example Clause N, wherein thepredetermined time period comprises a threshold amount of time aftercompletion of the multiplayer game session.

Example Clause P, the method of any one of Example Clauses M through O,further comprising: accessing trustworthiness data indicatingtrustworthiness values for identities that provide the plurality ofinstances of results data; and weighting the indications based at leastin part on the trustworthiness values.

Example Clause Q, the method of Example Clause P, further comprisingdetermining a trustworthiness value based at least in part on a resultshistory of a particular identity.

Example Clause R, the method of Example Clause Q, wherein thetrustworthiness value increases as a number of arbitrated winnerscorrectly indicated by the particular identity increases.

While Example Clauses M through R are described above with respect to amethod, it is understood in the context of this document that thesubject matter of Example Clauses M through R can also be implemented bya device, by a system, and/or via computer-readable storage media.

Example Clause S, a system, comprising: one or more processing units;and a computer-readable medium having encoded thereoncomputer-executable instructions to configure the one or more processingunits to: receive data representing a multiplayer game session; set athreshold number of indications based at least in part on a mode orcategory of the multiplayer game session; receive, as a set of resultsdata, a plurality of instances of results data from a correspondingplurality of computing devices, wherein an individual instance ofresults data discretely indicates an indication of an outcome for anindividual identity of a plurality of identities participating in themultiplayer game session; determine that a total number of indicationsfor an identity is greater than or equal to the threshold number ofindications; and in response to determining that the total number ofindications is greater than or equal to the threshold number ofindications, determine that the set of results data is valid, and store,in association with the identity and based at least in part on the validset of results data, data indicating that the identity is associatedwith an arbitrated outcome of the multiplayer game session.

Example Clause T, system of Example Clause S, wherein the thresholdnumber of indications is further based at least in part on a totalnumber of the plurality of identities participating in the multiplayergame session.

While Example Clauses S and T are described above with respect to asystem, it is understood in the context of this document that thesubject matter of Example Clauses S and T can also be implemented by adevice, via a computer-implemented method, and/or via computer-readablestorage media.

CONCLUSION

Although the techniques have been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the appended claims are not necessarily limited to the features oracts described. Rather, the features and acts are described as exampleimplementations of such techniques.

The operations of the example processes are illustrated in individualblocks and summarized with reference to those blocks. The processes areillustrated as logical flows of blocks, each block of which canrepresent one or more operations that can be implemented in hardware,software, or a combination thereof. In the context of software, theoperations represent computer-executable instructions stored on one ormore computer-readable media that, when executed by one or moreprocessors, enable the one or more processors to perform the recitedoperations. Generally, computer-executable instructions includeroutines, programs, objects, modules, components, data structures, andthe like that perform particular functions or implement particularabstract data types. The order in which the operations are described isnot intended to be construed as a limitation, and any number of thedescribed operations can be executed in any order, combined in anyorder, subdivided into multiple sub-operations, and/or executed inparallel to implement the described processes. The described processescan be performed by resources associated with one or more device(s) suchas one or more internal or external CPUs or GPUs, and/or one or morepieces of hardware logic such as FPGAs, DSPs, or other types ofaccelerators.

All of the methods and processes described above may be embodied in, andfully automated via, software code modules executed by one or moregeneral purpose computers or processors. The code modules may be storedin any type of computer-readable storage medium or other computerstorage device. Some or all of the methods may alternatively be embodiedin specialized computer hardware.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are understood within thecontext to present that certain examples include, while other examplesdo not include, certain features, elements and/or steps. Thus, suchconditional language is not generally intended to imply that certainfeatures, elements and/or steps are in any way required for one or moreexamples or that one or more examples necessarily include logic fordeciding, with or without user input or prompting, whether certainfeatures, elements and/or steps are included or are to be performed inany particular example. Conjunctive language such as the phrase “atleast one of X, Y or Z,” unless specifically stated otherwise, is to beunderstood to present that an item, term, etc. may be either X, Y, or Z,or a combination thereof.

Any routine descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode that include one or more executable instructions for implementingspecific logical functions or elements in the routine. Alternateimplementations are included within the scope of the examples describedherein in which elements or functions may be deleted, or executed out oforder from that shown or discussed, including substantiallysynchronously or in reverse order, depending on the functionalityinvolved as would be understood by those skilled in the art. It shouldbe emphasized that many variations and modifications may be made to theabove-described examples, the elements of which are to be understood asbeing among other acceptable examples. All such modifications andvariations are intended to be included herein within the scope of thisdisclosure and protected by the following claims.

What is claimed is:
 1. An arbitration system comprising: one or moreprocessing units; and a computer-readable medium having encoded thereoncomputer-executable instructions to configure the one or more processingunits to: determine that a third-party gaming system hosting amultiplayer game session is not certified for determining an outcome ofthe multiplayer game session; in response to determining that thethird-party gaming system is not certified for determining the outcomeof the multiplayer game session, receive, as a set of results data, aplurality of instances of results data from a corresponding plurality ofcomputing devices, wherein an individual instance of results dataprovides an indication of an outcome for an individual identity of aplurality of identities participating in the multiplayer game session;associate a title of the multiplayer game session with a category ofmultiplayer games; determine that a total number of indications for anidentity is greater than or equal to a threshold number of indications,wherein the threshold number of indications is determined based at leastin part on (i) a total number of the plurality of identitiesparticipating in the multiplayer game session and (ii) the category ofmultiplayer games; and in response to determining that the total numberof indications is greater than or equal to the threshold number ofindications, determine that the set of results data is valid, and store,in association with the identity and based at least in part on the validset of results data, data indicating that the identity is an arbitratedwinner of the multiplayer game session.
 2. The arbitration system ofclaim 1, wherein the computer-executable instructions further configurethe one or more processing units to, in response to determining that thetotal number of indications is not greater than or equal to thethreshold number of indications, generate a notification indicating thatthe set of results data is invalid data that is unable to be used todetermine the arbitrated winner of the multiplayer game session.
 3. Thearbitration system of claim 1, wherein the computer-executableinstructions further configure the one or more processing units to:determine that at least one computing device provides late results datareceived outside of a predetermined time period; and ignore the lateresults data when determining that the set of results data received fromthe plurality of computing devices indicates the total number ofindications for the identity is greater than or equal to the thresholdnumber of indications.
 4. The arbitration system of claim 3, wherein thepredetermined time period comprises a threshold amount of time aftercompletion of the multiplayer game session.
 5. The arbitration system ofclaim 1, wherein the threshold number of indications increases as areliability ratio increases, wherein the reliability ratio comprises atotal number of reliable indications received from the plurality ofcomputing devices over a total number of identities participating in themultiplayer game session.
 6. The arbitration system of claim 1, whereinthe computer-executable instructions further configure the one or moreprocessing units to expose performance data to a plurality ofthird-party systems via one or more application programming interfaces(APIs).
 7. The arbitration system of claim 1, wherein the thresholdnumber of indications is determined based at least in part on alikelihood of a consensus amongst the plurality of identities given thecategory of the multiplayer games.
 8. The arbitration system of claim 1,wherein the computer-executable instructions further configure the oneor more processing units to use the data indicating that the identity isthe arbitrated winner of the multiplayer game session to determine askill rating of the identity.
 9. The arbitration system of claim 1,wherein the computer-executable instructions further configure the oneor more processing units to: provide an opportunity for the plurality ofcomputing devices to send the plurality of instances of results databased, at least in part, on the determination that the third-partygaming system hosting the multiplayer game session is not certified fordetermining the outcome of the multiplayer game session.
 10. Thearbitration system of claim 1, wherein the computer-executableinstructions further configure the one or more processing units to:access trustworthiness data indicating trustworthiness values foridentities that provide the plurality of instances of results data; andweight the indications based at least in part on the trustworthinessvalues.
 11. The arbitration system of claim 10, wherein thecomputer-executable instructions further configure the one or moreprocessing units to determine a trustworthiness value based at least inpart on a results history of a particular identity.
 12. The arbitrationsystem of claim 11, wherein the trustworthiness value increases as anumber of arbitrated winners correctly indicated by the particularidentity increases.
 13. A method performed by an arbitration system,comprising: determining that a third-party gaming system hosting amultiplayer game session is not certified for determining an outcome ofthe multiplayer game session; in response to determining that thethird-party gaming system is not certified for determining the outcomeof the multiplayer game session, receiving, as a set of results data, aplurality of instances of results data from a corresponding plurality ofcomputing devices, wherein an individual instance of results datadiscretely indicates an indication of an outcome for an individualidentity of a plurality of identities participating in the multiplayergame session; associating a title of the multiplayer game session with acategory of multiplayer games; determining, by one or more processingunits, that a total number of indications for an identity is greaterthan or equal to a threshold number of indications, wherein thethreshold number of indications is determined based at least in part on(i) a total number of the plurality of identities participating in themultiplayer game session and (ii) the category of multiplayer games; andin response to determining that the total number of indications isgreater than or equal to the threshold number of indications,determining that the set of results data is valid, and storing, inassociation with the identity and based at least in part on the validset of results data, data indicating that the identity is an arbitratedwinner of the multiplayer game session.
 14. The method of claim 13,further comprising: determining that at least one computing deviceprovides late results data received outside of a predetermined timeperiod; and ignoring the late results data when determining that the setof results data received from the plurality of computing devicesindicates the total number of indications for the identity is greaterthan or equal to the threshold number of indications.
 15. The method ofclaim 14, wherein the predetermined time period comprises a thresholdamount of time after completion of the multiplayer game session.
 16. Themethod of claim 13, further comprising: accessing trustworthiness dataindicating trustworthiness values for identities that provide theplurality of instances of results data; and weighting the indicationsbased at least in part on the trustworthiness values.
 17. The method ofclaim 16, further comprising determining a trustworthiness value basedat least in part on a results history of a particular identity.
 18. Themethod of claim 17, wherein the trustworthiness value increases as anumber of arbitrated winners correctly indicated by the particularidentity increases.
 19. An arbitration system, comprising: one or moreprocessing units; and a computer-readable medium having encoded thereoncomputer-executable instructions to configure the one or more processingunits to: set a threshold number of indications based at least in parton a category of multiplayer games; determine that a third-party gamingsystem hosting a multiplayer game session is not certified fordetermining an outcome of the multiplayer game session; in response todetermining that the third-party gaming system is not certified fordetermining the outcome of the multiplayer game session, receive, as aset of results data, a plurality of instances of results data from acorresponding plurality of computing devices, wherein an individualinstance of results data discretely indicates an indication of anoutcome for an individual identity of a plurality of identitiesparticipating in the multiplayer game session; associate a title of themultiplayer game session with the category of multiplayer games;determine, based at least in part on associating the title of themultiplayer game session with the category of multiplayer games, that atotal number of indications for an identity is greater than or equal tothe threshold number of indications; and in response to determining thatthe total number of indications is greater than or equal to thethreshold number of indications, determine that the set of results datais valid, and store, in association with the identity and based at leastin part on the valid set of results data, data indicating that theidentity is associated with an arbitrated outcome of the multiplayergame session.
 20. The arbitration system of claim 19, wherein thethreshold number of indications is further based at least in part on atotal number of the plurality of identities participating in themultiplayer game session.